# Rotary position embedding

ModernBERT is a modern bidirectional encoder-only Transformer model (BERT-style), pre-trained on 2 trillion tokens of English and code data, with a native context length of up to 8,192 tokens.

E5-RoPE-Base is an embedding model based on rotary position embedding (RoPE), designed to support long-context retrieval tasks.

EVA02 is a Vision Transformer model pre-trained on ImageNet-22k through masked image modeling, suitable for image classification and feature extraction tasks.

This model incorporates rotary position embedding technology, is pre-trained and fine-tuned on 960 hours of LibriSpeech data sampled at 16kHz, and is suitable for English speech recognition tasks.